Brightness and color correction of image data of a line camera

ABSTRACT

A method for brightness and color correction of image data of a line camera is disclosed, wherein, for the detecting of image data with at least two line arrays of the line camera while illuminating a detection area of the line camera with an illumination module, a gray-scale image and at least two single-color images are recorded, and the image data is corrected with the help of a brightness function of the illumination module which is dependent on a line position of the line array and/or a distance of recorded objects. The brightness function is determined for the illumination module in advance and independent of the line camera and is stored in the illumination module, and the brightness function is read out by the line camera and used for the respective correction of the gray-scale image and the single-color images.

The present invention relates to a method for brightness and colorcorrection of image data of a line camera and to a camera which appliesthis method.

The image data of a line camera is used in many industrial and logisticapplications. This means that various processes can be automated.Besides diverse measuring, manipulation and inspection tasks, theautomatic sorting of objects based on codes is known and these are readwith the help of the image data. For this purpose, barcodes and varioustwo-dimensional codes are read, such as a Maxi-code or an Aztec-code, oralso labels which are decoded using text recognition (OCR).

In a typical application situation, such as on a production line,baggage handling in airports, or automated sorting of packages inlogistic centers, the objects are conveyed past the camera and imagedata of the objects having the codes arranged on them is obtained byscanning them line by line. The individual image lines are put togetherbased on the known or measured belt speed. Line cameras achieve a highresolution and speed.

Often, conventional line cameras record a monochrome image, also knownas a gray-scale or black-and-white image. With it, the best photon yieldand therefore the best signal-to-noise ratio is achieved, and colordetection is of no interest for many evaluations including the readingof codes that anyhow have only light and dark areas. If however, colorinformation is to be obtained, matrix cameras are normally referred to.However, this has disadvantages especially for fast belt applications,because a high frame rate is required, and the stitching of theirindividual images is computationally very intensive compared to thesimple lining up of image lines. In addition, a matrix image sensorcannot achieve the same number of pixels in the line direction as a lineimage sensor.

The most common way to configure a matrix image sensor for color captureis to provide pixel by pixel respectively two green, one red and oneblue filter in a Bayer pattern. However, there are also alternativecolor patterns that, for example, add a white channel (RGBW) or usesubtractive primary colors, such as red, yellow and blue (RYBY).

Color line cameras are also known. For example, they have three lines inred, green and blue (RGB), or an alternating arrangement of theseprimary colors on a single line or imitate the Bayer pattern by havingone line of alternating red and blue pixels and a second line of puregreen. The disadvantage of all these color image sensors is thatreceived light is lost through the color filters and therefore, ablack-and-white image remains superior for applications having highresolution requirements such as code reading. Therefore, the colorinformation can only be obtained at the cost of poorer performancecompared to a black-and-white evaluation.

Furthermore, a line camera is subject to dynamic variation in itsapplication situation. Dependencies on ambient light are largelyeliminated by an active illumination and, if necessary, by shielding.However, the illumination itself and the image recording are stillalways variable from object to object. This is mainly due to therespective object distance and inhomogeneities such as vignetting in theline direction. To compensate for these, it is known to measure anintegrated illumination and to teach-in brightness dynamics on-site.However, this leads to a large calibration effort. In addition, thebrightness dynamics and non-uniform spectral characteristics of theillumination are much more disturbing in a color recording because therecorded colors shift as a result, but this has not yet been compensatedfor.

DE 102015 116 568 describes color detection through the combination of amonochrome camera with a tunable color filter. However, as a result, toomuch time is lost in recording and with respect to a monochrome camera,a level of reception is missing due to the color filter.

In EP 3 012 778 A2 a double line camera is described which is driven bya multi-spectral lighting. This is effectively nothing more than also acolor filter that is now generated elsewhere but does not prevent theloss of time and light.

From US 2010/0316291 A1 and US 2012/0002066 A1 a respective code readingdevice is known that records images with a pixel matrix. Most of thepixels are monochrome and colored pixels are interspersed in a gridarrangement. These colored pixels form a Bayer pattern.

DE 20 2019 106 363 U1 discloses a code reader for reading optical codeswhich uses at least a double line as an image sensor of which, receivingpixels of at least one line are sensitive to white light and receivingpixels of the remaining line are sensitive to only one color. This, thenallows a gray-scale image and a color image to be recorded. In someembodiments, a primary color is reconstructed from the gray-scale imageand two other primary colors. However, the color correction isincomplete, and the color image still shows noticeable deviations incolor.

Therefore, it is the object of the present invention to improve therecording of color image data.

This object is achieved by a method for brightness and color correctionof image data of a line camera, sometimes also referred to as line scancamera, and by a camera, in particular a line camera or line scancamera, which uses this method. The line camera comprises a line-shapedimage sensor having a plurality of line arrays of light-receivingpixels. By a plurality of line arrays is meant a small number of atleast less than ten. The resolution in the line direction and thereforethe number of light-receiving pixels in the line direction is orders ofmagnitude higher and is several hundred, thousand or more. While anillumination module illuminates a detection area of the line camera, agray-scale image and at least two single-color images are recorded. Asingle-color image is an image of a specific color, for example blue orred. For this purpose, the light-receiving pixels have differentspectral sensitivities, in particular due to color filters or theomission thereof. Groups of light-receiving pixels with a certainspectral sensitivity can each be considered as part of a recordingchannel, a mono channel for the gray-scale image and several colorchannels for the single-color images.

The image data is corrected with a brightness function of theillumination module. The brightness function depends on up to twovariables, on the line direction along a line array designated forexample by X, or on the distance direction of the recorded objectsdesignated for example by Z, or in particular depends preferably onboth. In discretized form, the brightness function may be represented bya brightness vector or a brightness matrix. The brightness function maybe used directly for correction, or it may be incorporated into morecomplex corrections, which will be described below with reference toadvantageous embodiments.

The invention is based on the basic idea of determining in advance thebrightness function, for example in the final production of theillumination module, and to store it in the illumination module. Thiscan even be done completely independent of the line camera since at thispoint a connection between a particular line camera and the illuminationmodule is preferably not even known. Later in operation, the line cameratogether with the illumination module is used to then read out thebrightness function from the illumination module, and the line camerauses it directly for brightness and color correction, or the line cameragenerates its own correction function from the brightness functionstored in the illumination module and uses it in further operation. Thebrightness function is used for a correction in the mono channel and thecolor channels respectively, i.e. on the gray-scale image and thesingle-color images. A brightness correction in a color channel isautomatically also a color correction, since it shifts the mutualweighting of the colors. In further advantageous embodiments, the monochannel is included for a reconstruction of colors so that also itsbrightness has influence on the color.

Additional advantageous color corrections may be added, which also willbe described further on.

The invention has the advantage that a gray-scale or black-and-whiteimage is detected in full resolution with high contrasts and the bestpossible signal-to-noise ratio. At the same time, color information isobtained that may be used for various additional or alternativeevaluations. This additional color detection is not at the expense ofthe resolution or the signal-to-noise ratio of the gray-scale image.According to the invention, it is possible to compensate theillumination spectrum of the illumination module. The illuminationmodule is initially independent of the camera and can be calibratedindependently. This facilitates the commissioning or maintenance, as thecamera adapts itself to the illumination module used with it. There isno need to carry out teaching-in of the illumination characteristics onsite, which would involve some effort and expert knowledge.

Depending on the embodiment, the distribution of the light-receivingpixels with their respective color-selective light sensitivity over theline arrays of the line camera varies. Preferably, the light-receivingpixels of the mono channel for the gray-scale image form at least onecomplete line, which may be referred to as a white line. Theselight-receiving pixels are sensitive to white light, which should meanthat they perceive the entire optical spectrum and for example, have nocolor. Of course, the unavoidable hardware limitations of the camerapixels used constitutes the limit of the received light. Thedistribution of the light-receiving pixels of a color channel and thusthe recording of a respective single-color image with sensitivity foronly one color may form different patterns, which differ depending onthe embodiment. Line arrays for the color channels may be referred to asa color line as opposed to the white line. In general, white pixels mayonce again be interspersed in a color line, but the white line is initself responsible for such image information.

Preferably, two, three or four line arrays are provided, of which one ortwo line arrays are white lines. The said numbers here are exact and nota minimum requirement. With few line arrays, a particularly compactstructure of the image sensor is achieved. The minimum embodiment is adouble line with one white line and one color line, wherein the colorline then supports at least two color channels by a pattern oflight-receiving pixels with respective sensitivity for one of at leasttwo different colors. To achieve a higher resolution of the color imagein the line direction, at least two color lines are preferably provided.

The light-receiving pixels within a color line are preferably sensitiveto the same color. In other words, an entire color line is uniform, forexample a red line or a blue line. Thus, the corresponding colorinformation is detected in full resolution. The light-receiving pixelswithin a color line may also be sensitive to different colors, inparticular, in an alternating sequence such as red-blue-red-blue.Furthermore, it is conceivable to combine together uniform color linesand mixed color lines.

The gray-scale image is preferably used for reading codes, i.e. thecontent encoded in the code is read out. The gray-scale image has thefull resolution and the best possible signal-to-noise ratio, so thatcode reading is possible with the same quality as with a conventionalmonochrome line camera. At the same time, color information can beprovided without affecting the decoding result. This color informationmay be used for arbitrary functions, but also for functions related tocode reading, for example, an initial segmentation or the finding ofcode regions.

A color image is preferably generated from the single-color images. Acolor image in conventional language is an image with colors as normallyrecognizable by the human eye such as RGB, and is distinguished from asingle-color image that contains, for example, only the red colorinformation. If all the primary colors are detected as single-colorimages, it is sufficient to combine them into the color image. Besides,it is conceivable to reconstruct one of the primary colors from theothers. Particularly preferred is the use of the color image in thecontext of code reading and for its support, in order to identify,classify and/or distinguish from the image background code-carryingobjects and/or code regions. Often, the code underground differs incolor from the surrounding area, or the color information may be used toidentify a code-carrying object and separate it from the background.Alternatively, the color image is used for some other function, inparticular it is output as such and is then only used downstream, be itfor visualization and diagnostic functions or for completely differentadditional tasks. Thus, the two functions of recording a gray-scaleimage particularly suitable for code reading and a color image usefulfor supporting this or for other purposes, are combined in a method anda device, whereby the primary function of code reading is not impairedby the additional color image recording. The gray-scale image may beused additionally or alternatively for purposes other than code reading.

Preferably, a gray-scale image and a color image are generated. Thereby,both images are available for the decoding or other functions. The colorimage preferably has a lower resolution than the gray-scale image. Thehigh-resolution gray-scale image of the white line is already availablefor evaluations that are demanding in terms of resolution, such as codereading. The lower resolution may already occur originally due to toofew or too many light-receiving pixels of a respective color in the linedirection. Alternatively, a binning or down-sampling takes place at thehardware or software level.

Preferably, image lines recorded during a relative movement between theline camera and the objects or codes to be recorded are combined to anoverall image. Alternatively, in the case of bar codes, a code readingfrom a single line-shaped recording is conceivable, however also barcodes are preferably read from such a combined two-dimensional overallimage.

The brightness function is preferably modified each time by means of acolor normalization function for the color of the single-color image, sothat the correction for the gray-scale image and the single-color imagesis respectively carried out with its own brightness function, whereby acolor normalization function for different line positions and distancessets the brightness of the illumination module in its color inproportion to the brightness across the entire spectrum. The colornormalization function reproduces the spectral differences of theillumination module preferably in the same dimensions of line directionX and/or distance direction Z as the brightness function. In discretizedform, the color normalization function can be represented as a matrix orvector. The resolution of the color normalization function and thebrightness function may be different, which can then be adjusted, forexample by interpolation. If the color normalization function and thebrightness function are mixed per color channel, the result is acolor-adapted and normalized improved or modified brightness function.The brightness function for the mono channel or the gray-scale imagedoes not need any normalization, a corresponding gray normalizationfunction would only consist of ones and would not change anything, sincehere the gray value for the brightness would be in relation to itselfover the whole spectrum. The modification of the brightness functionwith the color normalization functions is preferably carried out duringthe commissioning, after the camera has read out the brightness functionfrom the illumination module, and thereafter, the modified brightnessfunction is stored in the line camera for using for brightness and colorcorrection in further operation.

The color normalization function is preferably determined in advancegenerally for the type of illumination module. In this embodiment, it isassumed that the series used, or the type of illumination module atleast shows a reasonably stable spectral performance across devices. Thecolor normalization functions are determined once, not per illuminationmodule. Deviations are accepted as tolerances. These color normalizationfunctions may be stored either in the illumination module or in the linecamera, since they do not depend on the specific illumination module,but only on its type.

The color normalization functions are preferably determined individuallyfor the illumination module in advance. In this alternative embodiment,the color normalization functions are likewise device-specific as in thebrightness function and are also stored in the illumination module forlater use in the line camera. Preferably, the color normalizationfunctions, and the brightness function are learned-in in the sameprocess, for example by using color-sensitive light receivers during thecalibration measurement of the brightness function in different X and/orZ directions.

The brightness function is preferably refined based on opticalparameters of the line camera. The brightness function stored in theillumination module preferably has only a low resolution of, forexample, around the ten level in the X and/or Z direction. This reducesthe effort required in the calibration of the illumination module if thebrightness function is to be initially stored there. Later, at theoperating location, the line camera reads-in this still rather coarselyresolved brightness function and converts it into a higher resolutionbrightness function using an optical model in which optical parameterssuch as focal length, aperture and the like are entered. Preferably, thebrightness function is first mixed with the color normalizationfunctions to obtain a brightness function per mono channel and colorchannel, and this brightness function obtained for each channel issubsequently refined. Since the brightness functions are already thennormalized, the same algorithm for refinement may be used in the monochannel and in the color channels. However, it is also conceivable touse an algorithm for refinement adapted for color per se or even foreach color in the color channels.

The gray-scale image and the single-color images are preferably recordedwith different analog and/or digital gains. Due to the color filters,the level in the color channels is typically lower, and this can becompensated for by an amplification. In this case, all colors can beincreased with the same amplification factor or are amplifieddifferently among one another. If the hardware allows, the best signalquality is achieved by analog amplification. Digital amplification maybe considered by a point-by-point scaling with the desired amplificationfactor in the brightness function or in the color normalizationfunction.

Preferably, two single-color images are recorded in two from threeprimary colors. Here, there is one mono channel and two color channelseach of a different primary color. Accordingly, the colored lines of theline camera have light-receiving pixels that are respectively sensitiveto one of the two primary colors and there are no light-receiving pixelsthat are sensitive to the third primary color. The primary colors arethe additive primary colors red, green and blue or the subtractiveprimary colors blue-green (cyan), purple (magenta) and yellow. In thatonly two of these are provided, light-receiving pixels and line arraysare spared. Alternatively, it would be conceivable to have all threerespective primary colors (RGBW, CMYW).

The third primary color is preferably reconstructed from the gray-scaleimage and the two single-color images. The white line records asuperposition of all primary colors, so that the third primary color canbe isolated when the other two primary colors are recorded.Nevertheless, with naive direct subtraction the colors would bedistorted beyond recognition, especially if the spectrum of theillumination module is inhomogeneous. However, according to theinvention, the colors are maintained thanks to the explained correctionwith the brightness function preferably taking into account colornormalization functions. An additional advantageous color correctionwill be explained in the following.

The two primary colors are preferably red and blue. In general, additiveprimary colors lead to better results. Especially green, which isprovided twice in the Bayer pattern, is not recorded in this preferredembodiment, so that for this color no light-receiving pixels and linearrays need to be provided. If required, green is generated from thewhite line and the red and blue color information. Graphically, green isreconstructed from G=3*W-R-B, whereby this leads only to satisfactoryresults with the brightness and color correction according to theinvention. The selection of the primary colors red and blue isparticularly advantageous if the illumination module has a weakerintensity in the green spectrum.

Preferably, corrected color values of a color image are formed fromlinear combinations of respective gray values of the gray-scale imageand single-color values of the single-color images with color correctingweighting factors. The weighting factors are static and are determinedempirically in order to achieve a good color impression with thisadditional color correction. In this particular context, the imageinformation detected with the white line is also considered as color andis therefore also included in the linear combinations, in case a primarycolor is reconstructed from the white line or rather the gray-scaleimage.

Preferably, corrected RGB values R′G′B′ are formed as a gray-scale imagewith gray values W, a red image with red values R and a blue image withblue values B are formed as

R′=x ₁ *R+x ₂*(3*W-R-B)+x ₃ *B+x ₄,

G′=x ₅ *R+x ₆*(3*W-R-B)+x ₇ *B+x ₈ and

B′=x ₉ *R+x ₁₀*(3*W-R-B)+x ₁₁ *B+x ₁₂

with weighting factors x₁ . . . x₁₂. This is a simple and clearcomputation instruction, whose manageable number of weighting factors x₁. . . x₁₂ is at the same time determined with reasonable effort and isflexible enough for a good color correction. Here, again, the preferredpair of single-color images in red and blue is recorded. For otherprimary colors, the above equations could be analogously specified withsubstitutions. Individual or a number of the weighting factors may bezero, especially the offsets x₄, x₈, x₁₂.

The corrected color values are preferably determined with a neuralnetwork.

Therefore, no weighting factors need to be determined manually, butrather this is learned-in automatically based on sample data. Thenecessary data sets with the color requirements which are learned-in ascorrect, may be specified via reference patterns. Particularlypreferred, is that the training is carried out based on color imagesfrom at least one additional color-sensitive or preferablycolor-calibrated sensor. The additional color sensor is required onlyonce, for example at the manufacturing site, so that its one-off costsare not so significant, and it generates in an easy manner labeledtraining data of the required quality and quantity.

The method according to the invention may be further embodied in asimilar manner and showing similar advantages. Such advantageousfeatures are described by way of example, but not exhaustively, in thedependent claims following the independent claims.

The invention is explained in more detail below also with respect tofurther features and advantages by way of example with reference toembodiments and with reference to the accompanying drawing. The figuresin the drawing show in:

FIG. 1 a schematic sectional view of a line camera;

FIG. 2 a three-dimensional view of an application of the line camera infixed mounting above a conveyor belt with objects, in particular forcode reading;

FIG. 3 a schematic representation of a line-shaped image sensor with ared line, a blue line and a white line;

FIG. 4 a schematic representation of a line-shaped image sensor with ared line, a blue line and two white lines;

FIG. 5 a schematic representation of a line-shaped image sensor with analternating red-blue line and a white line;

FIG. 6 a schematic representation of a line-shaped image sensor with twoalternating red-blue lines and two white lines;

FIG. 7 an exemplary flow chart for the generation of normalizedgray-scale and single-color images;

FIG. 8 an exemplary color normalization matrix for red;

FIG. 9 an exemplary color normalization matrix for blue;

FIG. 10 example images at different distances before and afternormalization for the mono channel of the gray-scale image;

FIG. 11 example images at different distances before and afternormalization for the red-color channel of the red image;

FIG. 12 example images at different distances before and afternormalization for the blue-color channel of the blue image;

FIG. 13 an exemplary spectrum of an illumination module for differentdistances; and

FIG. 14 an exemplary illustration of the quantum efficiency oflight-receiving pixels for different colors.

FIG. 1 shows a very simplified block diagram of a line camera 10, whichis preferably configured as a code reader for reading one- ortwo-dimensional optical codes. The line camera 10 detects received light12 from a detection area 14 through a photographic lens 16, representedhere only by a simple lens. A line-shaped image sensor 18 generatesimage data of the detection area 14 and the objects and code regions asnecessary that are present there. The image sensor 18 has at least twolines 20 a-b of light-sensitive receiving pixels 22, whereby in the linedirection, preferably a plurality of hundreds, thousands or even morereceiving pixels 22 are provided.

The image data of the image sensor 18 is read out by a control andevaluation unit 24. The control and evaluation unit 24 is implemented inone or more digital components, for example microprocessors, ASICs,FPGAs or the like, which may also be provided in whole or in partoutside the line camera 10. A preferred part of the evaluation is to puttogether detected image lines as an overall image. Otherwise, during theevaluation, the image data may in preparation be filtered, smoothed,tailored to specific areas or binarized. According to the invention, abrightness or color correction is provided, which will be explained inmore detail further on in reference to FIGS. 7 to 14. In a preferredembodiment of the line camera 10 as a code reader, a segmentation istypically performed in which individual objects and code regions arefound. The codes in these code regions are then decoded, that is, theinformation contained in the codes is read out.

In order to illuminate the detection area 14 sufficiently brightly withtransmitted light 26, an illumination module 28 having a light source30, typically a plurality of light sources such as in the form of LEDsas well as transmission optics 32 is provided. The illumination module28 is shown in FIG. 1 within a housing 34 of the line camera 10. This isa possible embodiment in which the illumination module 28 is insertedinto a suitable slot of the line camera 10 later in production or eveninto the finished device after production, for example at the site ofoperation. Alternatively, the illumination module 28 has its own housingor is an external device and is connected to the line camera 10 foroperation.

Data can be output at an interface 36 of the line camera 10, namely,read code information as well as other data in various processingstages, such as raw image data, pre-processed image data, identifiedobjects or code image data not yet decoded. On the other hand, it ispossible to parameterize the line camera 10 via the interface 36 or afurther interface.

FIG. 2 shows a possible application of the line camera 10 mounted on aconveyor belt 38 that conveys objects 40 in a conveying direction 42 asindicated by the arrow, through the detection area 14 of the line camera10. The objects 40 may carry code regions 44 on their outer surfaces.The task of the line camera 10 in this example application as a codereader, is to identify the code regions 44, read out the codes attachedthere, decode them and assign them to the respective associated object40. In order to also identify code regions 46 attached to the side,preferably several line cameras 10 having different perspectives areused. Additional sensors may be added, for example, an upstream laserscanner for detecting the geometry of the objects 40 or an incrementalencoder for detecting the speed of the conveyor belt 38. Stationarymounting of the line camera 10 on a conveyor belt 38 with objects 40 isalso conceivable in image evaluation applications other than codereading.

The detection area 14 of the line camera 10 is a plane with aline-shaped reading field corresponding to the line-shaped image sensor18. Accordingly, the illumination module 28 generates a line-shapedillumination area that, apart from tolerances, corresponds to thereading field. In FIG. 2, the illumination module 28 is shown simply andpurely schematically as a block within the line camera 10. As mentionedabove, the illumination module 28 may be an external device. Byrecording line by line the objects 40 in the conveying direction 42, anoverall image of the objects 40 which have conveyed past, together withthe code regions 44, is gradually formed. The lines 20 a-b lie so closeto one another that they practically detect the same object section.Alternatively, an offset can also be computationally compensated for.

The line camera 10 detects with its image sensor 18, on the one hand, agray-scale image or a black-and-white image that is used for codereading. In addition, color information or a color image is alsoobtained. The color information may be used for a variety of additionalfunctions. One example is the classification of objects 40, for exampleto find out whether it is a package, an envelope or a bag. It can bedetermined if a conveyor belt container is empty, such as the tray of aconveyor-tray or a box. Segmentation of the image data into objects 40or code regions 44 can be performed based on, or supported by, the colorinformation. Additional image recognition tasks may be solved, such asthe recognition of specific imprints or labels, for example forhazardous goods labeling, or fonts can be read (OCR, Optical CharacterRecognition).

FIGS. 3 to 6 show some examples of embodiments of the image sensor 18for such detection of black-and-white images and color information.Common to these embodiments is that at least one of the lines 20 a-d isa white line whose receiving pixels 22 detect light across the wholespectrum within the limits of the hardware. At least one other line 20a-d is a color line whose receiving pixels 22 are only sensitive to aparticular color, in particular due to appropriate color filters. Thedistribution of colors over the respective receiving pixels 22 of thecolored lines differs depending on the embodiment but deviates from theusual RGB and in particular from a Bayer pattern. Providing at least onecomplete white line is preferred because it allows a full resolutiongray-scale image to be recorded. In addition, a separation into whiteand colored lines is clearer. In general, however, differing patterns ofwhite and colored receiving pixels 22 mixed among the lines 20 a-d areconceivable. The respective receiving pixels 22 of the same spectralsensitivity are combined in a mono channel for the gray-scale image orin a respective color channel for a single-color image, for example forred-sensitive receiving pixels 22 in a red-color channel for a red imageand for blue-sensitive receiving pixels 22 in a blue-color channel for ablue image.

FIG. 3 shows an embodiment with one red line 20 a, one blue line 20 band one white line 20 c each. The lines 20 a-c are therefore homogeneousand the receiving pixels 22 within a line 20 a-c are sensitive to thesame optical spectrum. FIG. 4 shows a variation with an additional whiteline 20 d.

In the embodiment shown in FIG. 5, receiving pixels 22 sensitive to redand blue are alternately mixed within a color line 20a. Thus, incombination with a white line 20 b, a structure with a total of only twolines is possible. FIG. 6 shows a variation in which both the color line20 a-b and the white line 20 c-d are doubled.

While for code reading, the high resolution of the white line isdesired, in many cases the color information is only needed in a lowerresolution. Therefore, a certain loss of resolution in the colored linesas in FIGS. 5 and 6 may, under the circumstances, not be disturbing atall. In some cases, it is even conceivable to artificially reduce theresolution by merging pixels (binning, down-sampling) and thus improvethe signal-to-noise ratio.

These examples are only a selection based on the primary colors red andblue with white (RBW). Further embodiments use other color filters andcolors. Thus, also the use of green with red or blue (RGW, BGW) or allthree primary colors (RGBW) would be conceivable. Furthermore, thesubtractive primary colors blue-green (cyan), purple (magenta) andyellow in analogous combinations may also be considered (CMW, CYW, MYWor CMYW).

The raw image data of the different colored receiving pixels 22 are inmany respects too unbalanced to provide colors that can be used. This isfirstly due to the spatial detection situation, since an object 40 at agreat distance and at the edge of the lines 20 a-d is exposed to adifferent illumination intensity than a close, central object 40.Accordingly, there is a spatial dependence in an X-direction of thelines 20 a-d and in a Z-direction of the object distance. Moreover, theillumination module 28 has spectral characteristics in which the levelsof brightness in the different wavelength ranges differ significantlyfrom each other, especially when using semiconductor light sources suchas LEDs. Furthermore, the spatial and spectral characteristics acrossthe individual illumination modules 28 are scattered due to, forexample, batch differences of the light sources 30 and other tolerances.In the following, various advantageous embodiments describe a brightnessand color correction that compensates for the individual fluctuations ofthe illumination module 28 and/or general spectral and spatialfluctuations.

FIG. 7 shows an exemplary flow chart for the generation of corrected ornormalized gray-scale and single-color images, whereby, without beinglimited to this example, a mono channel for the gray-scale image and twocolor channels for a red image and a blue image will be described.

The illumination module 28 is calibrated independently of the linecamera 10, for example during final production, in order to be able toflexibly take into account its individual characteristics due totolerances, batch differences and the like. For example, theillumination module 28 in the production is measured on a sliding table,whereby a number of light-receiving elements or photodiodes distributedlaterally, i.e. in the X direction, respectively provides a brightnessvalue for the respective (X, Z) position of the photodiode while beingmoved at different distances from the illumination module 28. Thisresults in a brightness matrix which, for example, has a resolution of10x10, i.e. measurements were made at ten distances with ten laterallydistributed photodiodes or, alternatively, one photodiode shiftedlaterally ten times per distance. The resolution may of course differ,in particular the same resolution in the X and Z direction is by nomeans necessary, but too few values result in an incompletecompensation, while too many values increases unnecessarily thecalibration effort.

The brightness matrix 48 of the illumination module 28 obtained inadvance in this way is stored in a preferably non-volatile memory of theillumination module 28 (EEPROM) and is a starting point of the flowchart in FIG. 7. For the actual application, preferably the illuminationmodule 28 is connected to the line camera 10 already at the operatinglocation. There is no need to determine beforehand which illuminationmodule 28 will be used in which line camera 10 since the two devices canflexibly make themselves known to each other.

As a first adjustment step, not shown in FIG. 7, in the line camera 10different gain factors may be used in the mono channel and in the twocolor channels, i.e. gain_(color,i)=k_(i) gain_(mono) with k_(i)>1. Adifferentiation of the color channels among each other is optional, i.e.k_(i)=k can be valid for all i color channels. In doing so, the monochannel and color channels already reach a similar dynamic range. If onthe hardware side it is possible with the image sensor 18, such asseparate white and color lines, the different gain that occurs isalready analog and thus achieves better signal-to-noise characteristics.Alternatively or additionally, digital gains are possible. Pure digitalgain factors may in a simplistic way be multiplied in the presentedcorrection matrices of the color channels.

For a brightness adjustment, the line camera 10 now reads out thebrightness matrix 48 stored in the illumination module 28 in a monochannel refinement 50. Using optical parameters such as focal length,aperture and the like, a refined mono channel brightness matrix 52 iscalculated which contains significantly more entries than the originalbrightness matrix 48. The mono channel brightness matrix 52 compensatesfor inhomogeneities in the illumination of this individual illuminationmodule 28 along the line axis or X-axis and along the Z-axis, due to thedecrease in intensity with increasing distance. In doing so, a whiteadjustment for the mono channel or the gray-scale image is achieved.

In the color channels, the spectral differences are also to be takeninto account. For this purpose, additional color normalization matrices54, 56 are used. Color normalization matrices 54, 56 have the samedimensions X, Z as the brightness matrix 48 but can differ in theirresolution, which is then compensated for, for example, byinterpolation. FIG. 8 shows an example of a color normalization matrix54 for red and FIG. 9 shows an example of a color normalization matrix56 for blue. To obtain these color normalization matrices 54, 56,spectrometer measurements of the illumination module 28 are performed,and then the ratio of the intensity in the respective color red or blueto the intensity over the entire spectrum is formed for each (X, Z)position. Graphically, a color normalization matrix 54, 56 indicates aspatially resolved distribution, which proportion of the total intensitythe associated color has. Preferably, color normalization matrices 54,56 are not determined individually for each illumination module 28, butrather once for a type or a series of illumination modules 28. They arethen known independent of production and may optionally be stored in theillumination module 28 or the line camera 10, for example as a table(LUT, look-up table).

In a combination step 58, the color normalization matrices 54, 56 aremixed with the brightness matrix 48 for each color channel. In addition,in a simple advantageous implementation, the individual entries can bemultiplied with each other, provided that all the matrices 48, 54, 56are or will be suitably normalized. Alternatively, a more complexcombined calculation is performed, which may also include a resolutionadjustment of the matrices 48, 54, 56.

The respective resulting compensation matrices are then subjected to acolor channel refinement 60. For this, the same algorithm may be used asin the mono channel refinement 50, or color-specific properties aretaken into account which modify the algorithm for the color channelscollectively or even for individual color channels. The result arerefined color channel brightness matrices 62, 64 for the blue or redcolor channels. In doing so, a white adjustment is now also achieved forthe color channels and thus the single-color images. The refinedbrightness matrices 52, 62, 64 only have to be calculated once, forexample, during commissioning or for a connection between anillumination module 28 and a line camera 10.

In the brightness correction in the mono channel and color channelsexplained with reference to FIG. 7, the brightness matrix 48 is recordedindependent of spectral characteristics, and color-specific adjustmentsare made by the color normalization matrices 54, 56. Alternatively, itis conceivable to record the brightness matrix 48 directly in differentcolors and thus store it in the illumination module 28. Then, differentbrightness matrices 48 are generated for the mono channel and each colorchannel. The information of the color normalization matrices 54, 56 isalready contained therein, and the combination step 58 may be omitted.For this purpose, in particular for the measurement of the illuminationmodule 28, light-receivers or photodiodes with appropriate color filterscan be used, instead of photodiodes sensitive to white light, as above.The color normalization is then actually performed individually for theillumination module 28 instead of, up till now, generally for a type ora series.

FIGS. 10 to 12 illustrate the result achieved so far of normalizedwhite, red and blue values. The figures are respectively structured inthe same way, whereby

FIG. 10 illustrates the monochrome channel, FIG. 11 the red channel andFIG. 12 the blue channel. Here, compared to the monochrome channel, thered and blue channels are increased beforehand by a gain factor of aboutthree. In the columns, the distance or Z-direction is varied. The upperline shows a raw image with the line position X on the X-axis andvarious consecutively recorded lines on the Y-axis. The second lineshows the corresponding result in the form of a normalized image. Thebottom line shows an average over the image lines of the raw imagecompared to an average over the image lines of the normalized image. Thesomewhat lighter drawn line of the normalized image runs at leastapproximately flat; thus, the normalization has leveled as desired theirregular progressions of the darker line of the raw image.

The image data normalized in this way may be used as input data forfurther color normalization and color reconstruction. FIG. 13 showsfirstly an exemplary illumination spectrum of an illumination module 28.A peak 66 for blue and a peak 68 for red are clearly recognizable. Themultiple lines result from the fact that the illumination spectrum isdistance-dependent due to the dispersion of the optics. FIG. 14 showscomplementary exemplary quantum efficiencies of color filters forreceiving-pixels 22 with a white characteristic curve 70, a bluecharacteristic curve 72, a green characteristic curve 74 and a redcharacteristic curve 76.

In a wavelength range of around 480 nm, a local minimum is found in theillumination spectrum of FIG. 13. According to the example in FIG. 14,this is where the transmission window of a green filter would betypically found. Therefore, a blue and a red color channel, whichprovide similar intensities on a white target, are preferably used andnot a green color channel. In this way, the dynamic range is betterutilized, and a better signal-to-noise ratio is achieved.

When a blue and a red color channel are chosen, image data is determinedin two primary colors only. If a representation of the color in RGBvalues is desired, the missing color green may be reconstructed from afunction f(W, R, B) and for the first time from G=3*W-R-B. However, thisis still not sufficient for a good color reproduction since theillumination spectrum is inhomogeneous and has a local minimum in thegreen wavelength range. A certain compensation has been made by thenormalizations described above. For a result that is as true to thecolor as possible, preferably correlations between R, B and W are nowdetermined and used. These are, for example, linear combinations of theform

R′=x ₁ *R+x ₂*(3*W-R-B)+x ₃ *B+x ₄

G′=x ₅ *R+x ₆*(3*W-R-B)+x ₇ *B+x ₈

B′=x ₉ *R+x ₁₀*(3*W-R-B)+x ₁₁ *B+x ₁₂

with correlation or weighting factors x₁ . . . x₁₂

The weighting factors x₁ . . . x₁₂ are empirically determined and arestatic. For color channels other than blue and red without green,appropriate corrections are possible.

The weighting factors allow for a color reproduction despite the localminimum in the green spectrum shown in FIG. 13. To illustrate, one canimagine that the line camera 10 is recording a green target. In the bluechannel, as in the red channel, almost no green light is allowedthrough, and the recorded intensity is close to zero. In the monochannel, see the exemplary white characteristic line 70 in FIG. 14, thelittle amount of existing green light allowed through, results in anintensity slightly above zero. A high value x₆ in combination withcorrected values x₅ and x₇ can reconstruct the green value G′. With analternative black target, no significant intensity would be detected inany channel, which does not change the factors x₅. x₇ in the equationfor G′, so that quite correctly, a green value close to zero isreconstructed. Here, it can be seen that the offset values x₄, x₈, x₁₂are reasonably chosen not to be too large or to be even at zero. With agray target, both color channels give a certain signal and reconstruct acertain green value G′, which in sum results in the RGB color gray asdesired.

Alternatively or in addition to the presented weighting factors, aneural network, in particular with multiple hidden levels, is used. Asan input, a raw or pre-corrected color vector is defined, and the neuralnetwork returns a corrected color vector. Such a neural network can betrained, for example, with an additional color sensor that specifies thecolors to teach-in in a supervised learning for training-images. Inaddition, algorithms or neural networks may be used to improve thesignal-to-noise behavior by taking into account the color values of theneighboring pixels.

What is claimed is:
 1. A method for brightness and color correction ofimage data of a line camera, wherein for detecting the image data withat least two line arrays of the line camera under illumination of adetection area of the line camera with an illumination module, agray-scale image and at least two single-color images are recorded andthe image data is corrected with the help of a brightness function ofthe illumination module which is dependent on a line position of theline array and/or a distance of recorded objects wherein the brightnessfunction is determined for the illumination module in advance andindependent of the line camera and is stored in the illumination module,and the brightness function is read out by the line camera and is usedfor the respective correction of the gray-scale image and thesingle-color images.
 2. The method according to claim 1, wherein theline camera is a line camera for code reading.
 3. The method accordingto claim 1, wherein the gray-scale image is used for the reading ofcodes.
 4. The method according to claim 1, wherein a color image isgenerated from the single-color images.
 5. The method according to claim4, wherein the color image is used in particular to identifycode-carrying objects and/or code regions, to classify them and/or todifferentiate them from the image background.
 6. The method according toclaim 1,wherein the brightness function is respectively modified by acolor normalization function for the color of the single-color image, sothat the correction of the gray-scale image and the single-color imagesis respectively carried out with its own brightness function, wherein acolor normalization function sets for different line positions anddistances the brightness of the illumination module for its color inproportion to the brightness over the entire spectrum.
 7. The methodaccording to claim 6, wherein the color normalization functions aredetermined in advance generally for the type of illumination module. 8.The method according to claim 6, wherein the color normalizationfunctions are determined in advance individually for the illuminationmodule, in particular with the brightness function.
 9. The methodaccording to claim 6, wherein the color normalization functions aredetermined in advance individually for the illumination module with thebrightness function.
 10. The method according to claim 1, wherein thebrightness function is refined based on optical parameters of the linecamera.
 11. The method according to claim 1, wherein the gray-scaleimage and the single-color images are recorded with different analogand/or digital gains.
 12. The method according to claim 1, wherein twosingle-color images are recorded in two of three primary colors.
 13. Themethod according to claim 12, wherein the third primary color isreconstructed from the gray-scale image and the two single-color images.14. The method according to claim 12, wherein the two primary colors arered and blue.
 15. The method according to claim 1, wherein correctedcolor values of a color image are formed from linear combinations ofrespective gray values of the gray-scale image and single-color valuesof the single-color images with color correcting weighting factors. 16.The method according to claim 15, wherein corrected RGB-values R′G′B′are formed as a gray-scale image with gray values W, a red image withred values R and a blue image with blue values B are formed asR′=x ₁ *R+x ₂*(3*W-R-B)+x ₃ *B+x ₄G′=x ₅ *R+x ₆*(3*W-R-B)+x ₇ *B+x ₈B′=x ₉ *R+x ₁₀*(3*W-R-B)+x ₁₁ *B+x ₁₂ with weighting factors x₁ . . .x₁₂
 17. The method according to claim 15, wherein the corrected colorvalues are determined with a neural network, which is learned-in basedon color images of at least one further color-sensitive sensor.
 18. Acamera which comprises a line-shaped image sensor with at least two linearrays of light-receiving pixels for recording image data, and a controland evaluation unit for processing the image data, wherein the linearrays form a mono channel whose light-receiving pixels are sensitive towhite light for recording a gray-scale image, and at least two colorchannels whose light-receiving pixels are respectively sensitive only tolight in the color of its color channel, wherein the control andevaluation unit is configured so as to correct the image data inbrightness and color according to the method of claim
 1. 19. The cameraaccording to claim 18, wherein the camera is a code reader for readingan optical code.